Variational Inference

What is Variational Inference

= a method in Bayesian statistics and machine learning for approximating complex probability distributions:

• Instead of trying to directly calculate a complicated posterior distribution $p(z\mid x)$ (often intractable), we pick a simpler family of distributions $q_{\varphi }(z)$ and try to find the one that is closest to the true posterior.

Why use Variational Inference

• the Bayesian context: the marginal likelihood $p(x)$, which is needed for computing the posterior via Bayesian inference, is often intractable (cannot compute exactly)
• Variational Inference can turn inference into optimization.

How it works

1. Choose a family of approximating distributions $q_{\varphi }(z)$ (e.g., Gaussian with mean and variance parameters)
2. Optimize parameters $\varphi$ to maximize the Evidence Lower Bound (ELBO):

3. Use gradient-based methods for optimization.

Applications

• Variational Autoencoders (VAE)
• Probabilistic Graphical Models